Robotics II

Poznan University of Technology, Institute of Robotics and Machine Intelligence

Laboratory 4: Cones detection using Camera sensor

Back to the course table of contents

In this course, you will train an object detection algorithm. Your object will be a cone.

Part I - train the cone detector

First, follow the interactive tutorial. It uses the YOLOv5 model for object detection and the MIT Driverless FS Team cones dataset. Remember to save exported onnx model from the final step.

Part II - build the inference pipeline

Now, your goal is to build the object detection pipeline to deal with cones position. Note that we can’t extract depth distance to object. Due to, in the course, we only estimate fake-distance based on mathematical calculations.

Tasks

1. Move the onnx model to the docker container. Set the absolute path to model in RoboticsII-FSDS/config/config.yaml.

2. Your workspace file is scripts/vision_detector.py. Review its contents. You can execute it using the command roslaunch fsds_roboticsII vision_detector.launch.

3. Our model require input image in special format. Fill the preprocess function inside YOLOOnnxDetector class. Steps:

   • transpose image from channel last representation to channel first (HWC -> CHW), eg. (608,608,3) -> (3,608,608).
   • swap color channels from BGR to RGB,
   • convert array to np.float32 and rescale image values to <0,1>,
   • add additional 0 axis (convert image to batch), eg. (608,608,3) -> (1, 3,608,608).

4. At the end of camera_detection_callback fuction, function, implement a loop over the boxes that draw it on im array.

5. Uncomment steering node inside vision_detector.launch. Run the script again. Observe the behaviour of racecar and object detection. Try to increase the performance of steering:

   1. you can modify vehicle parameters in the config,
      
   2. you can modify dummy-steering cut off cone_too_small_thresh param,
      
   3. you can modify model postprocessing params: conf_thres or iou_thres.

As a result, upload screenshot from the rviz tool to the eKursy platform.